This invention concerns a computer network system and procedures for building and/or synchronising a second database from/with a first database. In particular, the invention concerns those computer network systems in which a first, already existing database is to be transferred into a second database which is to be newly constructed. In complex systems with one or more front end stations/applications and a back end, migrations traditionally take place in such a way that first the front end is migrated and only then the back end. In practice, simultaneous migration of both the front end and the back end is often not indicated, for various reasons (high complexity, long down time of system). Above all in the case of large DP projects in which a single-step migration (so-called “big bang”) from an existing database platform to the new database platform is ruled out, for a wide variety of reasons—e.g. because not all applications for access to the new database are yet completed, because for security reasons a full changeover to the new database is not yet indicated, because the operational behaviour of the new database still has to be investigated in detail, or similar—there is a need for a systematic approach which allows a controlled, gradual changeover from the existing database to the new database.
Furthermore, there is often the operational requirement to have the two databases in the practically consistent state at certain defined points in time, for instance at the end of the day. In other words, the data should be continuously kept synchronised on both database systems, and users should also be able to maintain the data, for instance using application software programs.
Since even after the initial transmission of the data from the first database to the second database (initial load), because of the continued maintenance of the first database, a very large number of changes of the data held in it can occur in a short time, approaches which are efficient regarding computing time and transfer cost (required communication bandwidth, incurred costs) are required. The demand on the system also increases if the changes are maintained online in the first database and are to be made available also in the second database as closely as possible in time (at least approximately in real time). In some cases, for collective or group changes, offline maintenance—at times of low operation—is also required and must be made possible.
Since the migration from the first database platform to the second database platform is generally carried out, as well as for application reasons (enterprise flow optimisation, enterprise restructuring, etc.) mostly from technical or IT points of view (faster access, more complex query options, change of hardware system platform, etc.), there are mostly considerable differences regarding the physical implementation, structures and organisational forms between the first and second databases. This aspect is particularly intensified if between the first and second databases there are structurally considerable differences regarding system architecture (hardware, operating system, database design and database implementation). In this case, changes which are to be made in the first database (=changes, deletions of existing entries, creating and filling new entries) cannot be mapped in the same way, i.e. not identically (1:1) in the second database. Also, changes are often complex, that is they affect a first plurality of entries in the first database, but because of the different structures and organisational forms a different plurality of entries in the second database, or entering changes in different and/or additional fields in the second database. This circumstance too excludes immediate maintenance of the changes in the second database in the identical way as it takes place in the first database.
Finally, it must be taken into account that in the case of large DP projects, usually multiple computer program applications access and change the databases. This circumstance—particularly in the case of online systems which are quasi-concurrent regarding accesses—has considerable influence on the strategy for keeping the second database up to date.
Because of transit times of messages/data flows in the networks in which the two databases are included and/or by which the two database platforms are connected to each other, and other influences (file length, priorities, etc.) in real time or online environments or even mixed (real time and batch processing systems), it is not directly possible to ensure that the changes are made available to the application software programs which access the second database in exactly the same sequence as they are executed in the first database. In other words, when data is transferred from one database to the other database, it can be overtaken by data which was transmitted earlier. This has the unwanted consequence that an “older” change can reset the data of a “newer” change to the “old” value. Also, because of these effects, the problem can occur that records are not yet completely maintained in the second database, so that incompletely changed, and thus in the end false, data is made available to the application software programs which access the second database.
Not least, efforts must be made so that the quality, operability, performance etc. of the original database is not considerably—ideally not at all—limited by the migration process.